1. Field of the Invention
The present invention relates to a language processing method and apparatus applicable to a speech recognizer for recognizing Japanese or Korean speech which is pronounced either continuously or with a pause between bunsetsu-phrases. The present invention is also applicable to a Japanese word processor of the type in which a Japanese sentence written in Kana (Japanese alphabet) only is converted into a Japanese sentence written orthographically using both Kana and Kanji (Chinese character).
In the above language processing method, when a set of bunsetsu-phrase candidates having various starting and ending positions in the phonetic expression, that is, a bunsetsu-phrase lattice is given, then, a bunsetsu-phrase sequence which is optimum as a Japanese or Korean clause or sentence is selected from the candidate set taking both the reliability of each bunsetsu-phrase and the degree of dependency between two bunsetsu-phrases into consideration. The optimum dependency structure of the selected bunsetsu-phrase sequence as a Japanese or Korean clause or sentence, and the degree of acceptability of the dependency structure thus obtained are also calculated.
In the Japanese language and the Korean language, there are a class of words called "independent word" such as noun, verb, adjective, adverb and so on, and a class of words called "dependent word" such as auxiliary verb and particle. It is noted that a linguistic unit called "bunsetsu-phrase" in this specification is an independent word followed by some (possibly 0) suitable dependent words. An example of bunsetsu-phrase is a noun followed by a particle indicating a case. A suitably conjugated form of a verb followed by an auxiliary verb is another example of bunsetsu-phrase.
2. Description of the Prior Art
A selection of an optimum bunsetsu-phrase sequence from a bunsetsu-phrase lattice is an important problem which appears in various situations of the Japanese or Korean language processing. Let us consider, for instance, a Japanese word processor of the type in which a Japanese sentence written as non-segmented Kana string is converted into a sentence written orthographically using both Kana and Kanji. In order to obtain a good result of conversion, the following two kinds of ambiguity must be resolved.
(A) Ambiguity in segmenting the given Kana string into a sequence of substrings corresponding to bunsetsu-phrases, and
(B) ambiguity arising from the homonymity of each substrings obtained as a result of the above segmentation.
An ideal way of resolving the above ambiguity will be selecting an optimum sequence of bunsetsu-phrases, taking both the acceptability of the sequence as a Japanese sentence and the reliability of each bunsetsu-phrase into consideration, from the bunsetsu-phrase lattice decided by all the possible segmentation of the given Kana string and by all the possible bunsetsu-phrases corresponding to each segment. However, only enumeration method has been known to solve the above-mentioned problem, and since the enumeration method is not practicable because of its enormous amount of computation, the following non-optimal methods have been used.
(1) The two-bunsetsu-longest-coincidence method is a well known method of segmenting a Kana string into a sequence of bunsetsu-phrases. (Hiroshi Makino and Makoto Kizawa: "Automatic Segmentation for Transformation of Kana into Kanji", Trans. Inf. Proc. Soc. Japan, Vol. 20, No. 4, pp.337-345 (1979)). According to this method, the total length of two adjacent bunsetsu-phrases is used as a measure of goodness for segmentation, and the boundary between two segments, which permit the longest interpretation as two adjacent bunsetsu-phrases, is adopted as a segmenting point.
(2) Another method is the least BUNSETSU's number method (Kenji Yoshimura, Tooru Hitaka and Sho Yoshida: "Morphological Analysis of Non-Marked-Off Japanese sentence by the least BUNSETSU's number method", Trans. Inf. Proc. Soc. Japan, Vol. 24, No. 1, pp. 40-46 (1983)). In this method, a segmentation which yields a sequence of bunsetsu-phrases having the least number of bunsetsu-phrases is regarded as the best segmentation.
In the above two methods, the strategy to search for a good segmentation is based on a heuristics, and there is no clear reason for the definition of optimality employed. Furthermore, they have no function of resolving the ambiguity coming from the homonymity of each segment, so that the best they can do is to make a list of candidates of bunsetsu-phrases in the order of plausibility, leaving the selection of the appropriate one to the user just like a conventional Japanese word processor which accepts only a segmented Kana string input.
(3) There is also a method in which a given Kana string is first segmented by the above-mentioned two-bunsetsu-longest-coincidence method, and then the sequence of segments is parsed from the standpoint of dependency between bunsetsu phrases in order to find the syntactically best bunsetsu-phrase for each segment. (Hiroshi Makino and Makoto Kizawa: "An Automatic Transformation System of Non-Segmented Kana Sentences into Kanji-Kana Sentences and its Homonym Analysis", Trans. Inf. Proc. Soc. Japan, Vol. 22, No. 1, pp. 59-67 (1981)).
This method is superior to the above-mentioned methods (1) and (2) in that the structure of the Japanese language is utilized for selecting the bunsetsu-phrase sequence from the set of bunsetsu-phrase candidates. However, this method has a problem that because the two-bunsetsu-longest-coincidence method itself is a heuristic method, the optimality of the segmentation is not always ensured. Furthermore, the assumption employed for the sake of simplicity in processing that the dependency relation holds between the nearest bunsetsu-phrases as long as it does not violate the rule of dependency is not always satisfied in the actual situation.
(4) There also exists a method which is, in principle, close to selecting the best bunsetsu-phrase sequence from a bunsetsu-phrase lattice. (Yoshimitsu Oshima, Masahiro Abe, Katsuhiko Yuura and Nobuyuki Takeuchi: "A Disambiguation method in Kana-to-Kanji Conversion Using Case Frame Grammar", Trans. Inf. Proc. Soc. Japan, Vol. 27, No. 7, pp. 679-687 (1986)).
According to this method, however, the selection must be made by the enumeration method, which is computationally impossible. Therefore, the number of possible sequences of bunsetsu-phrases must be limited by using local information prior to global parsing, resulting in a loss of global optimality.
In speech recognition, we also encounter a similar problem of optimum bunsetsu-phrase sequence selection.
One conceivable way of recognizing continuously spoken Japanese or Korean speech will be detecting possible bunsetsu-phrase segments, recognizing them and then listing them as bunsetsu-phrase candidates. Usually, these segments are located at various time positions and overlapped with each other, and there exists ambiguity resulting from homonymity of each segment. Since, in addition to this ambiguity, there is another kind of ambiguity arising from the uncertainty of recognition, the resulting bunsetsu-phrase lattice is more complicated than the one in the afore-mentioned word processor case. In order to obtain the final recognition result, a selection of an optimum bunsetsu-phrase sequence from such a complicated bunsetsu-phrase lattice is necessary. However, only the enumeration method or its modifications have so far been available to solve the problem, therefore, development of a more efficient method has been desired.